Method and program for generating three-dimensional brain map

ABSTRACT

Disclosed is a method for generating a three-dimensional brain map, comprising the steps of: acquiring a brain magnetic resonance imaging (MRI) image of an object; segmenting the brain MRI image into a plurality of regions; generating a three-dimensional brain image of the object including the plurality of regions by using the segmented brain MRI image; and generating a three-dimensional brain map of the object capable of simulating a process of transferring electrical stimulation to the brain of the object based on properties of each of the plurality of regions included in the three-dimensional brain image, wherein the step of segmenting includes the step of acquiring a segmented brain MRI image of the object by inputting a brain MRI image of the object into a model learned by using a plurality of processed brain MRI images.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of International PatentApplication No. PCT/KR2018/010170, filed on Aug. 31, 2018 which is basedupon and claims the benefit of priority to Korean Patent Application No.10-2017-0115779 filed on Sep. 11, 2017. The disclosures of theabove-listed applications are hereby incorporated by reference herein intheir entirety.

BACKGROUND

Embodiments of the inventive concept described herein relate to a methodand program for generating a three-dimensional brain map.

A magnetic resonance imaging (MRI) system is a device which expresses anintensity of a magnetic resonance (MR) signal for a radio frequency (RF)signal generated by a magnetic field of a specific intensity in contrastto acquire an image for a tomographic portion of an object. For example,when an RF signal which resonates only specific atomic nuclei (e.g.,hydrogen atomic nuclei or the like) is instantaneously irradiated to theobject after the object is laid in a strong magnetic field and is thenstopped, an MR signal is emitted from the specific atomic nuclei, andthe MRI system may receive the MR signal and may acquire an MR image.The MR signal refers to an RF signal radiated from the object. A levelof the MR signal may be determined by a concentration of certain atoms(e.g., hydrogen or the like) included in the object, a relaxation timeT1, a relaxation time T2, and the flow of blood flows or the like.

The MRI system includes features different from other imaging devices.Unlike the imaging devices, such as a computerized tomography (CT)device, in which acquisition of an image depends on a direction ofdetection hardware, the MRI system may acquire a 2D image or a 3D volumeimage oriented toward any point. Furthermore, unlike a CT device, anX-ray device, a positron emission tomography (PET) device, and a singlephoton emission computed tomography (SPECT) device, the MRI system doesnot expose a radioactive ray to an object and an inspector, and mayacquire an image having a high soft tissue contrast to acquire aneurological image, an intravascular image, a musculoskeletal image, anoncologic image, and the like, in which it is important to clearlydescribe abnormal tissues.

Transcranial magnetic stimulation (TMS) is a non-invasive treatmentmethod for the nervous system, which may treat nervous disease withoutmediation or invasive treatment. The TMS may apply electricalstimulation to the object using a change in magnetic field.

In general, the TMS was treated in such a manner as to apply electricalstimulation to a stimulation point known on a clinical basis or on atheoretical basis or determine a stimulation position while a usergradually moves the stimulation position. Thus, it is difficult toreflect a type of a coil used for procedure or a difference in bodystructure between persons, and it is difficult to directly identify theeffect according to procedure.

Furthermore, a brain disease treatment method through anelectroencephalogram (EEG), capable of measuring an electrical activityaccording to the activity of the brain of the object, and electricalstimulation is widely used. However, there is a need for development ofa guide method for reflecting a shape of the head, which differs foreach person, in the EEG and the electrical stimulation like the TMS.

SUMMARY

Embodiments of the inventive concept provide a method and program forgenerating a three-dimensional brain map.

The technical objects of the inventive concept are not limited to theabove-mentioned ones, and the other unmentioned technical objects willbecome apparent to those skilled in the art from the followingdescription.

According to an exemplary embodiment, a method for generating athree-dimensional brain map may include acquiring a brain magneticresonance imaging (MRI) image of an object, segmenting the brain MRIimage into a plurality of regions, generating a three-dimensional brainimage of the object including the plurality of regions, using thesegmented brain MRI image, and generating a three-dimensional brain mapof the object, the three-dimensional brain map being capable ofsimulating a process of delivering electrical stimulation to the brainof the object, based on properties of each of the plurality of regionsincluded in the three-dimensional brain image. The segmenting mayinclude acquiring the segmented brain MRI image of the object byinputting a brain MRI image of the object to a model learned using aplurality of processed brain MRI images.

Furthermore, the processed brain MRI image may be an image obtained bylabeling each of a plurality of regions included in the processed brainMRI image. The learned model may be a model for receiving a brain MRIimage and outputting a segmented brain MRI image.

Furthermore, the generating of the three-dimensional brain map of theobject may include generating a three-dimensional stereoscopic imagecomposed of a plurality of meshes, the three-dimensional stereoscopicimage being capable of simulating a process of delivering electricalstimulation to the brain of the object, using the three-dimensionalbrain image of the object.

Furthermore, the generating of the three-dimensional brain map of theobject may include acquiring a physical characteristic of each of theplurality of regions for simulating a flow of current according toelectrical stimulation to the brain of the object. The physicalcharacteristic may include at least one of isotropic electricalconductivity and anisotropic electrical conductivity of each of theplurality of regions.

Furthermore, the acquiring of the physical characteristic may includeacquiring a conductivity tensor image for the brain of the object fromthe brain MRI image of the object and acquiring anisotropic electricalconductivity of each of the plurality of regions using the conductivitytensor image.

Furthermore, the brain MRI image of the object may include a diffusiontensor image. The acquiring of the physical characteristic may includeacquiring anisotropic electrical conductivity of each of the pluralityof regions using the diffusion tensor image of the object.

Furthermore, the method may further include, when specific electricalstimulation is applied to one point of the head of the object using thethree brain map, simulating a state where the specific electricalstimulation is propagated in the brain of the object.

Furthermore, the method may further include acquiring a stimulationtarget point to apply electrical stimulation on the brain of the objectand acquiring a position to apply electrical stimulation to the head ofthe object to apply electrical stimulation to the stimulation targetpoint, using the three-dimensional brain map.

Furthermore, the acquiring of the position to apply the electricalstimulation may include acquiring a recommended path for deliveringelectrical stimulation from the scalp of the object to the stimulationtarget point, using the three-dimensional brain map and acquiring theposition to apply the electrical stimulation to the head of the objectfrom the recommended path.

According to an exemplary embodiment, a computer program may be combinedwith a computer which is hardware and may be stored in acomputer-readable storage medium to perform the method for generatingthe three-dimensional brain map.

The other detailed items of the inventive concept are described andillustrated in the specification and the drawings.

BRIEF DESCRIPTION OF THE FIGURES

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawing(s) will be provided by the Office upon request and paymentof the necessary fee.

The above and other objects and features will become apparent from thefollowing description with reference to the following figures, whereinlike reference numerals refer to like parts throughout the variousfigures unless otherwise specified, and wherein:

FIG. 1 is a flowchart illustrating a method for generating athree-dimensional brain map according to an embodiment;

FIG. 2 is a flowchart illustrating a method for generating athree-dimensional brain map of an object and performing simulationaccording to an embodiment;

FIG. 3 is a drawing illustrating the result of performing segmentationof a brain MRI image;

FIG. 4 is a drawing illustrating an example of a connectedcomponent-based noise rejection method;

FIG. 5 is a drawing illustrating an example of a post-processing schemeusing hole rejection;

FIG. 6 is a drawing illustrating an example of a three-dimensional brainimage generated from a brain MRI image of an object;

FIG. 7 is a drawing illustrating an example of a diffusion tensor image;

FIG. 8 is a drawing illustrating an example of a simulation result;

FIG. 9 is a flowchart illustrating a TMS stimulation navigation methodaccording to an embodiment;

FIG. 10 is a drawing illustrating an example of a TMS procedure methodaccording to an embodiment;

FIGS. 11A and 11B are drawings illustrating a relationship between amagnetic field and an electric field applied to a brain of an object;

FIG. 12 is a drawing illustrating information visualizing a magneticvector potential according to a type of a coil for procedure;

FIG. 13 is a drawing illustrating an example of a method for calculatinga position and direction of a coil;

FIG. 14 is a drawing illustrating examples of visualizing a state whereelectrical stimulation induced from a magnetic field of a coil forprocedure is propagated in the brain of an object;

FIG. 15 is a flowchart illustrating a patch guide method according to anembodiment;

FIG. 16 is a drawing illustrating the result of simulating an electricalstimulation result according to an embodiment;

FIG. 17 is a drawing illustrating an embodiment of a method for matchingimages;

FIG. 18 is a drawing illustrating an example of a three-dimensional scanmodel acquired using a depth camera;

FIG. 19 is a drawing illustrating an example in which a computing deviceconnected with a depth camera module captures the head of an object andguides a doctor to a position for attaching a patch to the captured headof the object; and

FIG. 20 is a drawing illustrating a portable computing device and adepth camera module connected thereto.

DETAILED DESCRIPTION

Advantages, features, and methods of accomplishing the same will becomeapparent with reference to embodiments described in detail belowtogether with the accompanying drawings. However, the inventive conceptis not limited by embodiments disclosed hereinafter, and may beimplemented in various forms. Rather, these embodiments are provided toso that this disclosure will be through and complete and will fullyconvey the concept of the invention to those skilled in the art, and theinventive concept will only be defined by the appended claims.

Terms used in the specification are used to describe embodiments of theinventive concept and are not intended to limit the scope of theinventive concept. In the specification, the terms of a singular formmay include plural forms unless otherwise specified. The expressions“comprise” and/or “comprising” used herein indicate existence of one ormore other elements other than stated elements but do not excludepresence of additional elements. Like reference numerals designate likeelements throughout the specification, and the term “and/or” may includeeach of stated elements and one or more combinations of the statedelements. The terms such as “first” and “second” are used to describevarious elements, but it is obvious that such elements are notrestricted to the above terms. The above terms are used only todistinguish one element from the other. Thus, it is obvious that a firstelement described hereinafter may be a second element within thetechnical scope of the inventive concept.

Unless otherwise defined herein, all terms (including technical andscientific terms) used in the specification may have the same meaningthat is generally understood by a person skilled in the art. Also, termswhich are defined in a dictionary and commonly used should beinterpreted as not in an idealized or overly formal detect unlessexpressly so defined.

The term “unit” or “module”, as used herein, means, but is not limitedto, a software or hardware component, such as field-programmable gatearray (FPGA) or application-specific integrated circuit (ASIC), whichperforms certain tasks. However, the “unit” or “module” is not limitedto software or hardware. A “unit” or “module” may advantageously beconfigured to reside on the addressable storage medium and configured toexecute on one or more processors. Thus, a “unit” or “module” mayinclude, by way of example, components, such as software components,object-oriented software components, class components and taskcomponents, processes, functions, attributes, procedures, subroutines,segments of program codes, drivers, firmware, microcode, circuitry,data, databases, data structures, tables, arrays, and variables. Thefunctionality provided for in the components and “unit” or “module” maybe combined into fewer components and “unit” or “module” or furtherseparated into additional components and “unit” or “module”.

In the present specification, the “object” may include a human, ananimal, or a part of a human or animal. For example, the object may bean organ, such as the liver, the heart, the womb, the brain, a breast,or the abdomen, or a blood vessel. Furthermore, the “object” may includea phantom. The phantom may refer to a material having a volume which isapproximately the same as a density and an effective atomic number of anorganism, which may include a spherical phantom having propertiessimilar to the human body.

Furthermore, in the present specification, the “user” may be a medicaldoctor, a nurse, a medical laboratory technologist, a medical imagingexpert, or the like, as a medical expert, or may be a technician whorepairs a medical apparatus, but not limited thereto.

Furthermore, in the present specification, the “magnetic resonance (MR)image” may refer to an image of an object acquired using the nuclearmagnetic resonance principle.

Hereinafter, an embodiment of the inventive concept will be described indetail with reference to the accompanying drawings.

FIG. 1 is a flowchart illustrating a method for generating athree-dimensional brain map according to an embodiment. FIG. 3 is adrawing illustrating the result of performing segmentation of a brainMRI image. FIG. 4 is a drawing illustrating an example of a connectedcomponent-based noise rejection method. FIG. 5 is a drawing illustratingan example of a post-processing scheme using hole rejection. FIG. 6 is adrawing illustrating an example of a three-dimensional brain imagegenerated from a brain MRI image of an object.

The method shown in FIG. 1 shows operations, performed by a computer, intime series. The computer in the present specification may be used asthe meaning including a computing device including at least oneprocessor.

In operation S110, the computer may acquire a brain magnetic resourceimaging (MRI) image of an object.

In an embodiment, the computer may be a workstation connected with anMRI image acquisition device, which may directly acquire a brain MRIimage of the object from the MRI image acquisition device.

Furthermore, the computer may acquire a brain MRI image of the objectfrom an external server or another computer.

In the disclosed embodiment, the brain MRI image of the object may referto an MRI image acquired by capturing a head part including the brain ofthe object. In other words, the brain MRI image of the object may referto an MRI image including the skull and scalp of the object as well asthe brain of the object.

In operation S120, the computer may segment (partition) the brain MRIimage acquired in operation S110 into a plurality of regions.

In an embodiment, the computer may segment the brain MRI image acquiredin operation S110 for each part. For example, the computer may segmentthe brain MRI image acquired in operation S110 into the white matter,the gray matter, the cerebrospinal fluid, the skull, and the scalp, butthe types capable of segmenting the brain MRI image are not limitedthereto.

In an embodiment, the computer may input the brain MRI image of theobject to a model learned using a plurality of processed brain MRIimages to acquire a segmented brain MRI image of the object.

In an embodiment, the processed brain MRI image may be an image acquiredby labeling each of the plurality of regions included in the brain MRIimage. Furthermore, the learned model may be a model for receiving thebrain MRI image and outputting a segmented brain MRI image.

In an embodiment, the learned model may refer to a model learned usingmachine learning and may refer to a model particularly learned usingdeep learning.

In an embodiment, the learned model may be, but is not limited to, amodel including one or more batch normalization layers, an activationlayer, and a convolution layer.

In an embodiment, the learned model may be configured to include ahorizontal pipeline configured with a plurality of blocks which extracta high-level characteristic from a low-level characteristic of an MRIimage and a vertical pipeline which collects and segmentscharacteristics extracted by the horizontal pipeline and performsegmentation of MRI with relatively degraded image quality.

Referring to FIG. 3 , the result 300(b) of segmenting a brain MRI image300 a is shown.

In an embodiment, the computer may post-process the segmented result.

In an embodiment, the computer may perform connected component-basednoise rejection. The connected component-based noise rejection methodmay be used to improve the result of segmentation performed using aconvolution neural network (CNN).

Referring to FIG. 4 , an example of the connected component-based noiserejection method is shown.

The computer may remove the other components 402, except for a connectedcomponent which is the largest chunk, from a segmentation image 400 toacquire an improved segmentation image 410.

In an embodiment, the computer may perform hole rejection. The holerejection may be used to remove a hole which is one of errors ofconvolution neural network based segmentation.

Referring to FIG. 5 , an example of a post-processing scheme using thehole rejection is shown.

The computer may remove at least a portion of a hole 502 included in asegmentation image 500 to acquire an improved segmentation image 510.

In operation S130, the computer may generate a three-dimensional brainimage of the object, including the plurality of segmented regions, usingthe brain MRI image of the object segmented in operation S120.

Referring to FIG. 6 , a three-dimensional brain image 600 generated fromthe brain MRI image of the object is shown.

Furthermore, an example of generating the segmented three-dimensionalbrain image 610 of the object from a segmented two-dimensional brain MRIimage of the object is shown in FIG. 6 .

In operation S140, the computer may generate a three-dimensional brainmap of the object, which is capable of simulating a process ofdelivering electrical stimulation to the brain of the object, based onproperties of each of the plurality of regions included in thethree-dimensional brain image generated in operation S130.

A detailed method of generating the three-dimensional brain map of theobject and performing simulation using the generated brain map will bedescribed below with reference to FIG. 2 .

FIG. 2 is a flowchart illustrating a method for generating athree-dimensional brain map of an object and performing simulationaccording to an embodiment. FIG. 7 is a drawing illustrating an exampleof a diffusion tensor image. FIG. 8 is a drawing illustrating an exampleof a simulation result.

The method shown in FIG. 2 may correspond to an embodiment of a methodshown in FIG. 1 . Thus, although there are contents omitted inconjunction with FIG. 2 , contents described in conjunction with FIG. 1are also applied to the method shown in FIG. 2 .

In operation S210, a computer may generate a three-dimensionalstereoscopic image composed of a plurality of meshes, which is capableof simulating a process of delivering electrical stimulation to thebrain of an object, using a three-dimensional brain image of an object.

In an embodiment, the computer may generate a three-dimensionalstereoscopic image composed of a plurality of surface meshes, each ofwhich includes triangles or quadrangles.

In an embodiment, the computer may generate a three-dimensionalstereoscopic image composed of a plurality of volumetric meshes, each ofwhich includes tetrahedrons or hexahedrons.

Types of the meshes constituting the three-dimensional stereoscopicimage may be differently set according to a purpose of simulation.

In operation S220, the computer may acquire a physical characteristic ofeach of the plurality of regions for simulating a flow of currentaccording to electrical stimulation to the brain of the object.

In an embodiment, the physical characteristic acquired in obtained S220may include at least one of isotropic electrical conductivity andanisotropic electrical conductivity of each of the plurality ofsegmented regions.

In an embodiment, the isotropic electrical conductivity may be acquiredby assigning electrical conductivity known by an experiment to eachsegmented region.

For example, electrical conductivity known for each region of the brainis shown in Table 1 below.

TABLE 1 Region Electrical conductivity (S/m) White matter 0.126 Graymatter 0.276 Cerebrospinal fluid 1.65 Skull 0.01 Skin 0.465

The anisotropic electrical conductivity may implement anisotropy ofwhite matter fibers in the white matter of the brain.

In an embodiment, the anisotropic electrical conductivity may beacquired from a conductivity tensor image for the brain of the object.

For example, the computer may acquire a conductivity tensor image forthe brain of the object from the brain MRI image of the object and mayacquire anisotropic electrical conductivity of each of the plurality ofsegmented regions using the acquired conductivity tensor image.

In another embodiment, the brain MRI image of the object may include adiffusion tensor image, and the computer may acquire anisotropicelectrical conductivity of each of the plurality of segmented regionsusing the acquired diffusion tensor image.

Referring to FIG. 7 , an example of a diffusion tensor image 700 isshown.

It is known that an eigenvector of the diffusion tensor image isidentical to an eigenvector of the conductivity tensor. The computer mayacquire anisotropic electrical conductivity in the direction of a neuralfiber included in the diffusion tensor image. For example, the directionof the neural fiber may have high electrical conductivity, and adirection perpendicular to the neural fiber may have low electricalconductivity.

When specific electrical stimulation is applied to one point of the headof the object using the three-dimensional brain map, in operation S230,the computer may simulate a state where the specific electricalstimulation is propagated in the brain of the object.

In an embodiment, the computer may simulate the state where theelectrical stimulation is propagated in the brain of the object usingthe mesh image acquired in operation S210 and the physicalcharacteristic acquired in operation S220.

Referring to FIG. 8 , an example of the simulation result is shown.

The electrical stimulation capable of being applied to the head of theobject may include at least one of a magnetic field, an electric field,and current. When a magnetic field is applied to the head of the object,current induced by the magnetic field may be propagated in the brain ofthe object.

In an embodiment, the computer may acquire a stimulation target point toapply electrical stimulation on the brain of the object. The computermay acquire a position to apply electrical stimulation to the head ofthe object to apply the electrical stimulation to the stimulation targetpoint, using the three-dimensional brain map of the object.

For example, the computer may acquire a recommended path for deliveringelectrical stimulation from the scalp of the object and the stimulationtarget point, using the three-dimensional brain map of the object andmay acquire a position to apply electrical stimulation to the head ofthe object from the recommended path.

The method for calculating and providing the position and direction toapply the electrical stimulation to the brain of the object will bedescribed below.

FIG. 9 is a flowchart illustrating a transcranial magnetic stimulation(TMS) stimulation navigation method according to an embodiment. FIG. 10is a drawing illustrating an example of a TMS procedure method accordingto an embodiment. FIGS. 11A and 11B are drawings illustrating arelationship between a magnetic field and an electric field applied to abrain of an object. FIG. 12 is a drawing illustrating informationvisualizing the magnetic vector potential according to a type of a coilfor procedure. FIG. 13 is a drawing illustrating an example of a methodfor calculating a position and direction of a coil. FIG. 14 is a drawingillustrating examples of visualizing a state where electricalstimulation induced from a magnetic field of a coil for procedure ispropagated in a brain of an object.

The TMS stimulation navigation method shown in FIG. 9 shows operations,performed by a computer, in time series.

Referring to FIG. 10 , an example of the TMS procedure method is shown.

TMS is a treatment method of making a treatment coil 1000 close to oneside of the head of an object 10 and stimulating a specific portion ofthe brain using an electric field induced in the brain of the object 10by a magnetic field generated by the coil 1000.

A magnetic field generated around the treatment coil 1000 may vary inintensity and shape according to a shape of the treatment coil 1000. Anappearance in which an electrical signal is propagated may also varyaccording to a shape of the head and brain of the object 10.

Thus, according to the disclosed embodiment, the computer may calculateand provide a stimulation point according to a type of the coil 1000 andmay provide a simulation result according to the shape of the head andbrain of the object 10.

In operation S910, the computer may acquire a stimulation target pointto apply electrical stimulation on the brain of the object.

The stimulation target point may be selected on a clinical ortheoretical basis according to disease to be treated. In an embodiment,the stimulation target point be indicated using the three-dimensionalbrain image or the three-dimensional brain map of the object generatedby the disclosed embodiment.

In operation S920, the computer may acquire information about a spatialdistribution of a magnetic vector potential of a coil for TMS procedure.

In an embodiment, the information about the spatial distribution mayinclude information visualizing the magnetic vector potential using amagnetic dipole according to a shape of the coil for TMS procedure.

Referring to FIG. 12 , information 1210 and 1260 visualizing magneticvector potentials according to types of coils 1200 and 1250 forprocedure is shown.

In operation S930, the computer may acquire one or more parameters foracquiring an optimal stimulation condition for the stimulation targetpoint acquired in operation S910, from the spatial distribution acquiredin operation S920.

In an embodiment, the optimal stimulation condition for the stimulationtarget point may refer to a condition where an intensity of a magneticfield applied to the stimulation target point by the coil for TMSprocedure becomes maximum.

Referring to FIGS. 11A and 11B, a relationship between a magnetic fieldand an electric field applied to the brain of the object is shown.

Referring to a simulation image 1100 of FIG. 11A, images acquired byrespectively visualizing a magnitude of a magnetic field applied to thebrain of the object, a magnitude of a gradient (potential), and amagnitude of an electric field induced by the magnetic field are shown.The magnitude of the electric field applied to the brain of the objectmay be calculated by adding the magnetic field applied to the brain ofthe object and the gradient.

Referring to a graph of FIG. 11B, a correlation between the magneticfield applied to the brain of the object and the electric field inducedby the magnetic field is shown.

According to the graph of FIG. 11B, it may be seen that, as a strongermagnetic field is applied to the brain of the object, a strongerelectric field is induced in the brain of the object.

Thus, it may be seen that the maximum stimulation condition for thestimulation target point is a condition where the intensity of themagnetic field applied to the stimulation target point by the coil forprocedure becomes maximum.

In an embodiment, the parameter acquired by the computer may include anoptimal point having the highest magnetic vector potential value in thespatial distribution of the magnetic vector potential induced by thecoil.

Furthermore, the parameter acquired by the computer may include anoptimal vector which is a normal vector where multiplication with agradient at the optimal point becomes minimum among normal vectors wherethe optimal point is a start point.

Referring to FIG. 12 , optimal points 1212 and 1262 and optimal vectors1214 and 1264 of the magnetic vector potentials 1210 and 1250 are shown.

An optimal point (x, y, z) and an optimal vector v may be calculated byEquations 1 and 2 below.

$\begin{matrix}{\max\limits_{x,y,z}{f( {x,y,z} )}} & \lbrack {{Equation}\mspace{14mu} 1} \rbrack\end{matrix}$

In Equation 1 above, f denotes the magnetic vector potential map. Aposition (x, y, z) having the highest value in the magnetic vectorpotential map f may be calculated as an optimal point by Equation 1above.

$\begin{matrix}{\min\limits_{x,y,z}{{\nabla{f( {\overset{\_}{x},\overset{\_}{y}\;,\overset{¯}{z}} )}^{T}}{\nu( {x,y,z} )}}} & \lbrack {{Equation}\mspace{14mu} 2} \rbrack\end{matrix}$

In Equation 2 above, ∇f(x, y, z) denotes the value acquired bydifferentiating f used when defining the optimal point at the optimalpoint x, y, z, and v(x, y, z) denotes the normal vector in the directionof (x, y, z).

In step S940, the computer may calculate a position and direction of thecoil, which satisfy the optimal stimulation condition for thestimulation target point acquired in operation S910, using the parameteracquired in operation S930.

In an embodiment, the calculating of the position and direction of thecoil may include calculating the position and direction of the coil suchthat the stimulation target point is closest in the direction of theoptimal vector from the optical point.

Referring to FIG. 13 , an example of the method for calculating theposition and direction of the coil is shown.

When an object 10 and a stimulation target point S 12 of the object 10are acquired, the computer may determine one point 14 on the scalpclosest from the stimulation target point 12.

In this case, a distance between the stimulation target point 12 and theone point 14 on the scalp closest from the stimulation target point 12is D, and a vector where the point 14 is a start point and where thestimulation target point 12 is an end point is K. Furthermore, athickness of a coil 1310 is 2P.

The computer may generate and apply a matrix, shown in Equation 3 below,arranging a vector K 1320 and an optimal vector 1312 of the coil 1310.

$\begin{matrix}{{{R_{x}(\theta)} = \begin{bmatrix}1 & 0 & 0 \\0 & {\cos\;\theta} & {{- \sin}\;\theta} \\0 & {\sin\;\theta} & {\cos\;\theta}\end{bmatrix}}{{R_{y}(\theta)} = \begin{bmatrix}{\cos\;\theta} & 0 & {\sin\;\theta} \\0 & 1 & 0 \\{{- \sin}\;\theta} & 0 & {\;{\cos\;\theta}}\end{bmatrix}}{{R_{z}(\theta)} = \begin{bmatrix}{\cos\;\theta} & {{- \sin}\;\theta} & 0 \\{\sin\;\theta} & {\cos\;\theta} & 0 \\0 & 0 & 1\end{bmatrix}}} & \lbrack {{Equation}\mspace{14mu} 3} \rbrack\end{matrix}$

Thus, the location of the coil may be calculated as Equation 4 below.Locdipole=S+K*(D+P)  [Equation 4]

In operation S950, when placing the coil for procedure at the locationcalculated in operation S940 in the direction calculated in operationS940, the computer may simulate a state where electrical stimulationinduced from a magnetic field of the coil for procedure is propagated inthe brain of the object.

In an embodiment, the computer may perform simulation using thethree-dimensional brain map generated according to in the method shownin FIGS. 1 and 2 .

For example, the computer may acquire a brain MRI image of the objectand may generate the three-dimensional brain map of the object, which iscapable of simulating a process of delivering electrical stimulation tothe brain of the object, based on properties of each of a plurality ofregions included in the acquired brain MRI image.

The computer may simulate a state where electrical stimulation by thecoil is propagated in the brain of the object, using the generatedthree-dimensional brain map.

Furthermore, the three-dimensional brain map may include athree-dimensional stereoscopic image composed of a plurality of meshes,which is capable of simulating a process of delivering electricalstimulation to the brain of the object.

In an embodiment, the computer may visualize a state where electricalstimulation induced from a magnetic field of the coil for procedure ispropagated in the brain of the object, using the three-dimensionalstereoscopic image.

Referring to FIG. 14 , examples of visualizing a state where electricalstimulation induced from a magnetic field of the coil for procedure ispropagated in the brain of the object are shown.

In the disclosed embodiment, the computer may be connected with a robotarm device equipped with a coil for TMS procedure. The robot arm devicemay include a mechanical device capable of moving the coil for TMSprocedure to a position specified by the computer.

The robot arm device may automatically perform a procedure using the TMScoil for a patient depending on the result calculated by the computer bymoving the coil for TMS procedure to the position specified by thecomputer according to the disclosed embodiment.

FIG. 15 is a flowchart illustrating a patch guide method according to anembodiment. FIG. 17 is a drawing illustrating an embodiment of a methodfor matching images. FIG. 18 is a drawing illustrating an example of athree-dimensional scan model acquired using a depth camera. FIG. 19 is adrawing illustrating an example in which a computing device connectedwith a depth camera module captures a head of an object and guides alocation for attaching a patch to the captured head of the object. FIG.20 is a drawing illustrating a portable computing device and a depthcamera module connected thereto.

In the disclosed embodiment, a patch may include a brain stimulationpatch. For example, the brain stimulation patch may include, but is notlimited to, an electrical stimulation patch and an ultrasonicstimulation patch. Furthermore, the patch may include an EEG patch.Herein, the type of the patch according to the disclosed embodiment isnot limited to the above-mentioned examples.

In operation S1510, a computer may acquire a three-dimensional scanmodel including the head of an object using a depth camera.

The depth camera may include a 3-dimensional laser scanner of atriangulation technique, a depth camera using a structure beam pattern,a depth camera using a time-of-flight (TOF) technique using a reflectiontime difference of an infrared ray, and the like, but the type thereofis not limited thereto.

The depth camera may be used to acquire a 3-dimensional scan model byreflecting distance information in an image.

In an embodiment, an object, that is, a patient sits on a round stool,and a user, that is, a doctor locates the depth camera using a temporaryfixing device, such as, a tripod, such that the face of the patient isviewed well from a height of the face of the patient.

The doctor starts a scan using the depth camera, and acquires a3-dimensional scan model including the head of the patient by turningthe patient slowly one turn.

In an embodiment, the depth camera may be provided in a fixing module,which is automatically rotatable, and may rotate around the patientlocated in the center to acquire a 3-dimensional scan model.

Meanwhile, according to the disclosed embodiment, to facilitate a3-dimensional scan without separate high-priced equipment, a depthcamera module may be connected to a portable computing device (e.g., asmartphone, a tablet PC, or the like), the computing device, to whichthe depth camera module is connected, may be fixed using a temporaryfixing device, such as a tripod, which may be easily acquired, and thepatient may be rotated after he or she sits on a stool or the like toacquire a 3-dimensional scan model.

Referring to FIG. 20 , a portable computing device 2000 and a depthcamera module 2010 connected to the portable computing device 2000 areshown.

Furthermore, referring to FIG. 18 , an example of a 3-dimensional scanmodel 1800 acquired using a depth camera is shown.

In an embodiment, the computer may generate a 3-dimensional modelincluding the head of the object using a distance image collected usingthe depth camera, and may align and add images captured at differenttime points to reconstruct a 3-dimensional model of the object. Forexample, the computer may reconstruct a model by collecting3-dimensional data in the form of a point cloud from distance imagescollected using the depth camera. However, the method for generating the3-dimensional model is not limited.

In operation S1520, the computer may acquire a 3-dimensional brain MRImodel of the object.

In an embodiment, the acquiring of the 3-dimensional brain MRI model ofthe object may include acquiring a brain MRI image of the object andgenerating a 3-dimensional brain map of the object, which is capable ofsimulating a process of delivering electrical stimulation to the brainof the object, based on properties of each of a plurality of regionsincluded in the brain MRI Image of the object.

Moreover, the generating of the 3-dimensional brain map of the objectmay include generating a 3-dimensional stereoscopic image composed of aplurality of meshes, which is capable of simulating the process ofdelivering the electrical stimulation to the brain of the object.

The method for generating the 3-dimensional brain map, described inconjunction with FIGS. 1 to 8 , may be used as the method for acquiringthe 3-dimensional brain MRI model of the object by the computer inoperation S1520.

In operation S1530, the computer may match the 3-dimensional scan modelincluding the head of the object and the brain MRI model of the object.

Referring to FIG. 17 , an embodiment of matching images is shown.Referring to an image 1700 shown in FIG. 17 , a brain MRI picture of theobject and an image acquired by modeling a brain structure of the objectare overlapped with each other.

In the image 1700, the lower three images may correspond to an examplein which the brain MRI picture and the image acquired by modeling thebrain structure are not matched with each other. Furthermore, in theimage 1700, the upper three images may correspond to an example in whichthe brain MRI picture and the image acquired by modeling the brainstructure are matched with each other.

The computer may calculate a change generated in the brain of the objectby electrical or ultrasonic stimulation of a patch depending on alocation at which the patch is attached, using the brain MRI model.Furthermore, the computer may calculate a location to actually attachthe patch, using the 3-dimensional scan model including the head of theobject.

Thus, the computer may calculate a location to attach the patch to thehead of the object by matching the 3-dimensional scan model includingthe head of the object and the brain MRI model of the object, and maythus calculate a change generated in the brain of the object. Similarly,the computer may calculate a location to attach the patch to the head ofthe object to cause a specific change in the brain of the object and mayprovide the result.

In an embodiment, the matching by the computer may include calculatingfacial features of the scan model and the brain MRI model and matchingthe scan model and the brain MRI model using the facial features of thescan model and of the brain MRI model.

Because the scan model including the head of the object and the brainMRI model of the object differ in style from each other, it is difficultto match the two models. Thus, the computer may match the two modelsusing the facial feature of the object.

In an embodiment, the calculating of the facial feature of the scanmodel including the head of the object may include acquiring a colorimage and a depth image, each of which includes the head of the object,calculating the facial feature of the object using the color imageincluding the head of the object, and calculating a 3-dimensionallocation of the facial feature of the object using the depth imageincluding the head of the object.

Referring to FIG. 18 , an example of matching the scan model 1800including the head of the object and a brain MRI model 1810 of theobject to generate the matched model 1820 is shown.

In operation S1540, the computer may acquire an image by capturing thehead of the object using the depth camera.

For example, the doctor may move while directly carrying the temporarilyfixed depth camera such that the depth camera faces the head of thepatient.

In operation S1550, the computer may match one location of the imagecaptured in operation S1540 and one location on the matched model.

For example, when the computer captures one point of the head of theobject using the depth camera, it may calculate where the one pointbeing captured corresponds to on the matched model.

In an embodiment, the computer may match the captured image and thematched model and may display an image for guiding a user to thelocation of the patch to be attached to the head of the object.

Referring to FIG. 19 , a computing device 1900, to which a depth cameramodule is connected, may capture a head 1910 of the object. Thecomputing device 1900 may display an image for guiding a user to alocation 1920 for attaching a patch 1930 to the captured head 1910 ofthe object.

In an embodiment, the computing device 1900 may determine a location toattach the patch 1930 on the matched model and may display the location1920 corresponding to the location determined on the captured image.

Furthermore, the computing device 1900 may recognize the patch 1930 onthe captured image and may guide a user along a movement direction ofthe recognized patch 1930.

Furthermore, the computing device 1900 may determine whether therecognized patch 1930 is attached to the determined location 1920.

In an embodiment, at least one marker is attached to or displayed on thepatch 1930. For example, at least one of a specific figure, a specificcolor, and a specific 2-dimensional code may be attached to or displayedon the patch 1930, and the computing device 1900 may recognize the patch1930 using the marker attached to or displayed on the patch 1930 and maytrack movement of the patch 1930.

For example, when the doctor captures the head of the patient using thecomputing device 1900 or the depth camera connected to the computingdevice 1900 while changing the location of the head, the location of thehead of the patient displayed on the computing device 1900 may also bechanged and the location of the patch 1930 recognized by the computingdevice 1900 may also be changed. In this case, the computing device 1900may track the patch 1930 even when the computing device 1900 is moved toguide the doctor to attach the patch 1930 to an accurate location of thehead of the patient.

In an embodiment, the computing device 1900 may recognize the patch 1930on the captured image and may guide a user along a movement direction ofthe recognized patch 1930. For example, the computing device 1900 maydisplay the movement direction of the patch 1930 such that the patch1930 may be attached to the determined location 1920.

Furthermore, the computing device 1900 may determine whether therecognized patch 1930 is attached to the determined location 1920. Forexample, the computing device 1900 may determine whether a locationwhere the patch 1930 is finally recognized corresponds to the determinedlocation 1920. When the determined location 1920 and the location towhich the patch 1930 is attached differ from each other, the computingdevice 1900 may provide a notification for requesting to change thelocation of the patch 1930.

In an embodiment, the computing device 1900 may recognize the patch 1930attached to the head of the object on the captured image and maydetermine the location of the recognized patch 1930. The computingdevice 1900 may acquire a location on the matched model, whichcorresponds to the determined location of the patch 1930.

For example, when an EEG is performed, an EEG patch may be attached to aconsistent location regardless of the shape and structure of the head ofthe user, or an EEG patch may be attached to any location. In this case,it is difficult to know in detail whether the brain wave acquired by theEEG patch is a brain wave received from any direction of the brain ofthe object.

Thus, according to the disclosed embodiment, the computing device 1900may capture the head of the object, to which one or more EEG patches areattached, and may acquire the locations of the one or more recognizedEEG patches from the captured image.

The computing device 1900 may acquire a location on the matched model ofthe object, corresponding to the acquired location of the EEG patch, andmay determine in detail whether the brain wave acquired by the EEG patchattached to the head of the object is received from any portion of thebrain of the object. For example, the computing device 1900 may analyzea signal source of the brain wave received from each EEG patch utilizingthe disclosed embodiment.

FIG. 16 is a drawing illustrating the result of simulating an electricalstimulation result according to an embodiment.

Referring to FIG. 16 , a 3-dimensional model of a head 1600 of an objectand an embodiment in which a patch 1610 is attached to one location onthe 3-dimensional model are shown.

When the patch 1610 is attached to the one location of the 3-dimensionalmodel of the head 1600 of the object, a computer may simulate the resultof delivering electrical stimulation by the patch 1610 to the brain 1650of the object.

In an embodiment, the computer may acquire a 3-dimensional brain map forthe brain 1650 of the object and may determine a location of the patch1610 to be attached to the head of the object, using the 3-dimensionalbrain map.

In an embodiment, the determining of the location of the patch 1610 mayinclude acquiring a purpose of using the patch 1610, simulating aprocess of delivering electrical stimulation to the brain 1650 of theobject depending on the location at which the patch 1610 is attached tothe head 1600 of the object, and determining the location of the patch1610 using the acquired purpose and the simulation result.

For example, when desiring to apply specific stimulation to the brain1650 of the object, the computer may determine the location of the patch1610 at which specific stimulation may be applied to the brain 1650 ofthe object, using the simulation result.

The computer may match the location of the patch 1610 determinedaccording to the embodiment illustrated in FIG. 16 with one point of thehead of the object captured using the depth camera and may display animage for guiding the patch to the matched location.

Steps of the method or algorithm described in connection with anembodiment of the inventive concept may be directly implemented inhardware, may be implemented with a software module executed byhardware, or may be implemented by a combination of the hardware and thesoftware module. The software module may reside on a random accessmemory (RAM), a read only memory (ROM), an erasable programmable ROM(EPROM), an electrically erasable programmable ROM (EEPROM), a flashmemory, a hard disc, a removable disc, a CD-ROM, or any type ofcomputer-readable storage medium which is well known in the technicalfield to which the inventive concept pertains.

According to the disclosed embodiment, the computer may segment a brainMRI image using the previously learned model to segment the brain MRIimage automatically within a short time.

Thus, anyone may acquire a three-dimensional brain image of the objectwithin a short time in the medical field. In addition, the computer mayprovide a simulation effect capable of visually identifying the effectof electrical stimulation to the brain of the object.

Furthermore, the computer may calculate and provide a stimulation pointcapable of applying optimal stimulation to a stimulation target point byusing magnetic vector potential information according to a type of acoil for TMS procedure together, thus increasing the effect ofprocedure.

Furthermore, the computer may guide the doctor to the location of anelectrical stimulation path using head modeling and MRI modeling toguide the doctor to the location of the electrical stimulation path withregard to a head and a brain structure, which differ for each person.

The effects of the inventive concept are not limited by the abovedescribed effects, and other effects which are not described here may beclearly understood by those skilled in the art from the abovedisclosure.

While the inventive concept has been described with reference toexemplary embodiments, it will be apparent to those skilled in the artthat various changes and modifications may be made without departingfrom the spirit and scope of the inventive concept. Therefore, it shouldbe understood that the above embodiments are not limiting, butillustrative.

What is claimed is:
 1. A method for generating a three-dimensional brainmap, the method comprising: acquiring a brain magnetic resonance imaging(MRI) image of an object; segmenting the brain MRI image into aplurality of regions, wherein each region of the plurality of regionscorresponds to a particular part of a brain; generating athree-dimensional brain image of the object including the plurality ofregions, using the segmented brain MRI image; generating athree-dimensional brain map of the object, the three-dimensional brainmap being capable of simulating a process of delivering electricalstimulation to the brain of the object, based on properties of each partof the brain, wherein the segmenting comprises: acquiring the segmentedbrain MRI image of the object by inputting the brain MRI image of theobject to a machine learning model, wherein the machine learning modelperforms learning by using a plurality of processed brain MRI images,and outputs the segmented brain MRI image having the plurality ofregions corresponding to parts of the brain; acquiring information on aselected stimulation target point to apply the electrical stimulation onthe brain of the object, and indicating the stimulation target point onthe three-dimensional brain map; acquiring information about a spatialdistribution of a magnetic vector potential of a coil for delivering theelectrical stimulation to the brain of the object, wherein theinformation about the spatial distribution include informationvisualizing the magnetic vector potential using a magnetic dipoleaccording to a shape of the coil; acquiring one or more parameters foracquiring an optimal stimulation condition for the stimulation targetpoint, wherein the optimal stimulation condition for the stimulationtarget point is a condition where an intensity of a magnetic fieldapplied to the stimulation target point by the coil becomes maximum;calculating and providing a position and a direction of the coil, whichsatisfy the optimal stimulation condition for the stimulation targetpoint, by calculating the position and direction of the coil such thatthe stimulation target point is closest in the direction of an optimalvector from an optical point, which are calculated based on the acquiredone or more parameters for acquiring the optimal stimulation conditionfor the stimulation target point; and simulating a state where theelectrical stimulation induced from a magnetic field of the coil ispropagated in the brain of the object, and displaying a result of thesimulating.
 2. The method of claim 1, wherein the processed brain MRIimage is an image obtained by labeling each of a plurality of regionsincluded in the processed brain MRI image, and wherein the learned modelis a model for receiving a brain MRI image and outputting a segmentedbrain MRI image.
 3. The method of claim 1, wherein the generating of thethree-dimensional brain map of the object includes: generating athree-dimensional stereoscopic image composed of a plurality of meshes,the three-dimensional stereoscopic image being capable of simulating theprocess of delivering the electrical stimulation to the brain of theobject, using the three-dimensional brain image of the object.
 4. Themethod of claim 1, wherein the generating of the three-dimensional brainmap of the object includes: acquiring a physical characteristic of eachof the plurality of regions for simulating a flow of current accordingto the electrical stimulation to the brain of the object, and whereinthe physical characteristic includes at least one of isotropicelectrical conductivity and anisotropic electrical conductivity of eachof the plurality of regions.
 5. The method of claim 4, wherein theacquiring of the physical characteristic includes: acquiring aconductivity tensor image for the brain of the object from the brain MRIimage of the object; and acquiring anisotropic electrical conductivityof each of the plurality of regions using the conductivity tensor image.6. The method of claim 4, wherein the brain MRI image of the objectincludes a diffusion tensor image, and wherein the acquiring of thephysical characteristic includes: acquiring anisotropic electricalconductivity of each of the plurality of regions using the diffusiontensor image of the object.
 7. The method of claim 1, wherein theacquiring information on the selected stimulation target pointcomprises: acquiring a recommended path for delivering the electricalstimulation from the scalp of the object to the stimulation targetpoint, using the three-dimensional brain map; and acquiring the positionto apply the electrical stimulation to the head of the object from therecommended path.
 8. A non-transitory computer-readable recording mediumstoring a computer program for generating the three-dimensional brainmap, and configured to be coupled to a computer hardware, and theprogram includes instructions to execute the method of claim
 1. 9. Themethod of claim 1, wherein the segmenting comprises: segmenting thebrain MRI image into a white matter region, a gray matter region, acerebrospinal fluid region, a skull region, and a scalp region, andwherein the generating the three-dimensional brain map of the objectcomprises: simulating the process of delivering the electricalstimulation to the brain of the object, based on an electricalconductivity of the gray matter region, an electrical conductivity ofthe cerebrospinal fluid region, an electrical conductivity of the skullregion, and an electrical conductivity of the scalp region.